perf: trim the answer payload to what the client renders (round 2, phase 4)#482
Conversation
Round-2 phase 4. The final SSE event (and the non-stream answer JSON) shipped every source's full chunk content plus server-only context, landing as one large frame after the prose had already streamed — pure transfer/serialize tail on answer completion. toClientAnswerPayload (new answer-client-payload.ts) now trims sources at the ROUTE BOUNDARY only: - Drops adjacent_context, memory_cards, table_facts, index_unit, and document_summary — audited to have zero client consumers (generation packing and evidence building consume them server-side, before the trim). - Truncates content to 700 chars at a word boundary; the client reads content only as the snippet fallback (retrieval_synopsis ?? content, answer-render-policy.ts). retrieval_synopsis, source_metadata, document_labels, scoring, and identity fields pass through untouched. - rag.ts, both cache layers, eval behavior, and logAnswerDiagnostics see the full answer unchanged; sourceGovernanceWarnings still compute from the full sources before the trim (ordering documented at both call sites). - Trim is non-mutating, so cached answers keep full sources (unit-tested). A representative 8-source answer payload shrinks by more than half (unit-tested lower bound; real chunks are typically larger than the fixture). Verified: tsc, eslint (0 warnings), prettier, targeted vitest including new answer-client-payload suite (125 tests green), and full verify:ui — 124 passed / 0 failed with phases 0-4 stacked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
This pull request has been ignored for the connected project Preview Branches by Supabase. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0c68d07d9e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const trimmed: SearchResult = { | ||
| ...source, | ||
| content: truncateAtWordBoundary(source.content ?? "", clientSourceContentMaxChars), | ||
| }; |
There was a problem hiding this comment.
When an answer source is longer than 700 chars, this replaces source.content before it reaches the browser. The dashboard still computes safety findings from answer.sources[].content (src/lib/clinical-safety.ts:117-119, called from ClinicalDashboard.tsx:2771), not just snippets; if the only contraindication/dose-limit/monitoring phrase is past the cutoff and no quote card duplicates it, the Safety findings panel drops that warning. A focused test that runs toClientAnswerPayload before extractSafetyFindings with a phrase after char 700 would reproduce it.
Useful? React with 👍 / 👎.
… phase 3 — auth) (#481) * perf: skip the proxy session-refresh getUser for API routes Round-2 phase 3, auth part (isolated for security review). Every authenticated request previously paid two serial auth-server round trips: src/proxy.ts ran supabase.auth.getUser() for every matched path including /api/*, and the route handler then independently validated the caller via getOptionalAuthenticatedUser. Nothing consumed the proxy's result on API routes — the call existed only to keep session cookies fresh. The proxy now skips only the session-refresh getUser for /api/* paths: - API handlers keep validating the caller themselves (bearer token and/or @supabase/ssr cookie) — no authorization decision moves or weakens, and the handlers remain fail-closed exactly as before. - The CSP nonce header still applies to every matched response including /api/* (matcher unchanged). - Session cookies still refresh on every page navigation, and the browser Supabase client refreshes its own token (autoRefreshToken) for bearer-based API calls, so long-lived sessions behave as before. Deliberately NOT done: the trusted-internal-header design (proxy forwards a validated user id for handlers to trust) was rejected — handlers validating tokens independently is the stronger defense-in-depth posture. The legacy cookie-token extraction in supabase/auth.ts was left untouched: tests define legacy sb-access-token / plain-JSON cookie support as intended behavior. New tests pin the contract: getUser is skipped for /api/* (CSP still stamped), runs on page navigations with an sb- cookie, and never runs without one. Existing CSP nonce tests unchanged and green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf: trim the answer payload to what the client renders (#482) * fix: preserve cookie refresh and safety evidence --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 participant
Summary
Phase 4 of the round-2 performance initiative (stacked on #481). The final SSE event (and non-stream answer JSON) shipped every source's full chunk content plus server-only context — one large frame landing after the prose had already streamed, i.e. pure transfer/serialize tail on perceived answer completion.
toClientAnswerPayloadtrims sources at the route boundary only:adjacent_context,memory_cards,table_facts,index_unit,document_summarycontent→ 700 chars at a word boundary; the client reads it only as the snippet fallback (retrieval_synopsis ?? contentinanswer-render-policy.ts:186)retrieval_synopsis,source_metadata,document_labels, all scoring/identity fields,visualEvidence,smartPanel, quote cardsAudit notes: follow-up questions never POST sources back to the server; answer-thread storage keeps sources client-side for the same render paths, so trimmed sources round-trip safely.
Ordering & purity contract (documented at both call sites):
sourceGovernanceWarningsandlogAnswerDiagnosticsconsume the FULL answer before the trim; the trim is non-mutating so both cache layers keep full sources;rag.tsand eval behavior are byte-identical.Measured: a representative 8-source answer payload shrinks by >50% (unit-tested lower bound — real chunks are typically larger than the fixture).
Clinical governance preflight
npm run verify:cheapcomponents: lint (0 warnings), typecheck, targeted vitest green (full vitest on the stack passed in perf: batch image signed URLs + cache seeded catalog fetches (round 2, phase 2) #479; only additive test files since)npm run verify:ui— 124 passed / 0 failed (answer surface, source cards, citation jump, document viewer, drawers) with phases 0–4 stackednpm run format:check— cleaneval:retrieval:quality— not required: no retrieval, ranking, selection, chunking, or scoring behavior changed (trim happens after the answer is fully computed and cached)eval:rag/eval:quality— not required: answer generation, synthesis prompt, and post-processing untouched (server-side answer object is byte-identical; only the wire payload shrinks)source_metadatapasses through untrimmed)🤖 Generated with Claude Code